Neural Network-based Language Model for Conversational Telephone Speech Recognition

Preface This dissertation is the result of my own work and includes nothing which is the outcome of work done in collaboration except where specifically indicated in the text. I hereby declare that my thesis does not exceed the limit of length prescribed in the Special Regulations of the M. Phil. examination for which I am a candidate. The length of my thesis is 14980 words. Acknowledgements I would like to thank Professor Phil Woodland for his help and guidance over the course of this project. I would also like to thank David Mrva for providing a customized version of the LPlex tool capable of summing over words in a specified shortlist. Abstract This thesis presents a large scale neural network language model for telephone conversation transcriptions. By mapping n-gram contexts to a continuous vector space, the neural network is trained with softmax normalization to operate as a probability estimator. The smooth nature of the resulting distributions achieves consistently reduced perplexity for restricted subsets of the vocabulary. Excessive training time is a major issue and optimized linear algebra libraries are used for an efficient implementation of feed forward and back propagation during training. A word-class interpretation of the network inputs and outputs is demonstrated to obtain improved perplexity over the n-gram model when training data is limited.